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Abstract, The world has moved on since the introduction of the Distributed Interactive Simulation (DIS) standard in the early 1980s. 
The cold-war maybe over but there is still a requirement to train for and analyze the next generation of threats that face the free 
world. With the emergence of new and more powerful computer technology and techniques means that modeling and simulation 
(M&S) has become an important, and growing, part in satisfying this requirement. As an industry grows, the benefits from 
standardization within that industry grow with it. For example, it is difficult to imagine what the USA would be like without the 110 
volts standard for domestic electricity supply. This paper contains an overview of the outcomes from a recent workshop to 
investigate the possible future of M&S standards within the federal government 


1.0 INTRODUCTION 

Determining the origins of Modeling and 
Simulation (M&S) is a difficult task. With a 
little imagination it is possible to envision the 
architects of the Egyptian’s pyramids 
contemplating the momentous task ahead 
of them with the aid of a small scale model. 
Letting your mind wonder back in time a 
little further and it is not hard to image a 
caveman’s children throwing some rocks at 
a boulder. The skills the children learnt 
from this training enabled them to help 
defend off an approaching predator, if the 
need arose. Thus this might have been the 
first training simulator. 

Though it might be hard to determine the 
origins of M&S it is clear that its usage has 
increased over the last few decades due to 
the rise in computer technology. This 
increase in usage has enabled simulation to 
be applied in areas such as optimization, 
safety engineering, testing, training and 
education (Sokolowski and Banks, 2009). 

In the last decade, even computers games 
have been put to serious M&S applications 
(National Research Council, 1997), 


With more and more applications of M&S 
being produced, M&S developers have a 
rich source of historical simulations to look 
at to help solve their problems. Some of 
these solutions will be consider better than 
others and soon the better solutions will 
become standards for the industry. 

Sadly though the development of M&S 
standards is not that simple. There has 
been various issues with their development 
over the years. This paper explores some 
of the issues and challenges that are future 
M&S standards will have to face, 

1.1 Workshop 

This paper main source of information was 
from the outputs of the “Standards in 
Modeling and Simulation: The next 10 
years” workshop which was held at the 
Virginia Modeling, Analysis and Simulation 
Center (VMASC) on March 31 st 2010 until 
April 2 ,ld 2010. The workshop had 
approximately 60 attendees over the three 
days which represented various interested 
parties from academia, industry and 
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government. Though the workshop was 
intended to be focused on military modeling 
and simulation (M&S) standards, there were 
individuals from groups outside this arena at 
the workshop including NASA and the 
Society for Simulation in Healthcare. 

The purpose of the workshop was to give 
everyone involved an opportunity to think 
and discuss all aspects of M&S standards 
over the next 10 years. By conducting the 
meeting in a non-attributable environment, it 
allowed participants to engage in more 'out 
of the box' thinking without being concerned 
that their ideas would be attributable to 
themselves or their organization. 

1.2 Overview 

In the next section of this paper, the 
terminology and concepts of standards are 
introduced. Standards specific to M&S are 
then introduced in section 3.0. The issues 
and challenges for M&S standards are then 
discussed in section 4.0. Finally, conclusion 
to the paper is given in section 5.0. 

2.0 STANDARDS 

A formal definition of a standard is given by 
the Federal Office of Management and 
Budget circular (1998): 

“ (1) Common and repeated use of 
rules, conditions, guidelines or 
characteristics for products or 
related processes and production 
methods, and related management 
systems practices. 

(2) The definition of terms; 
classification of components; 
delineation of procedures; 
specification of dimensions, 
materials, performance, designs, or 
operations; measurement of quality 
and quantity in describing materials, 
processes, products, systems, 
services, or practices; test methods 
and sampling procedures; or 
descriptions of fit and measurements 
of size or strength. 


The term "standard" does not 
include the following: 

(1) Professional standards of 
personal conduct. 

(2) Institutional codes of ethics." 

This definition is no way definitive as there 
is dissatisfaction within the circle of those 
who it affects (Finkleman, 2007). As there 
was some ambiguity to the meaning of 
standards, the following list gives a more 
general indication of the different general 
types of standards. 

• De facto - standards that have achieved 
dominant position by public acceptance or 
market forces i.e. VHS vs, Betamax. A 
more formal definition of this type of 
standard was given above in part (1) of the 
Federal Office of Management and Budget 
circular (1998). 

• Voluntary - standards are formally 
proposed and accepted by a community of 
interest e.g. key furniture dimensions. In 
some cases compliance to a voluntary 
standard might be necessary to successfully 
participate in a particular market, for 
example optional standards requirements 
on M&S grants. 

• De Dejure - standards that are mandated 
by law i.e. residential building codes. 

For the purpose of this paper this list is 
deem adequate definition of standards 
though the list does not capture every 
aspect of a standard. For example there are 
implicit and explicit standards. An example 
of an implicit standard is a computer game’s 
action configuration on the buttons of a 
game consul controller, where the user 
would expect the ‘fire’ and ‘jump’ actions to 
be allocated to certain buttons. 

2.1 Why Standards? 

The key reason for having M&S standards 
are: cost savings, technical superiority and 
convergence. Cost savings come through 
standards enabling simulation reuse. 
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Technical superiority is gained by allowing 
existing technology to work more effectively 
i.e. simulations federations. Finally, 
standards allow for convergence of the M&S 
usage over the different application domains 
and thus promoting synergy between them. 

2.1.1 Requirement for future M&S 
standards 

With the rise of more powerful visualization 
tools for use within simulation, allowing 
M&S vendors to give a dazzling display of 
graphics to potential customers, there have 
been concerns about a charlatan aspect 
within the M&S industry. This concern was 
coined “Garbage in, Hollywood out” in 
Roman (2005). Thus Validation and 
Verification M&S standards have been 
proposed a means to counteract this trend. 

There is a requirement for other possible 
M&S standards, especially relating to data. 

It should be noted that some data standards 
already exist (i.e. SEDRIS), 

2.2 Standards Bodies 

For standards, other than De Facto 
standards, to come into existence there 
needs to be an organization that manages 
their development. For M&S standards, the 
main body that does this work is the 
Simulation Interoperability Standards 
Organization (SISO). SISO was formed in 
1989 and became a recognized standards 
development organization by the IEEE in 
2003. 

It is not surprising that M&S has its own 
standards organization, given the vast 
number of these organizations in the United 
States (U.S.). Figure 1 gives an overview of 
all these organizations. To give the reader 
an idea of scale of this chart, the box that 
has been circle in the lower-left corner is for 
Department of Defense standards. 



Standards and Conformity Assessment Bodies of Hie U.5. 


Figure 1 : Organizational chart of standards 
organization with the U.S. {Bipes t 2007) 

3.0 M&S STANDARDS 

Even when the focus is narrowed from 
standards to M&S standards, there remain 
many standards to consider. The list 
includes both those standards developed 
specifically for military M&S (e g,, the Test 
and Training Enabling Architecture, or 
TENA, for test range-oriented distributed 
simulation) and those developed for broader 
applications that have been applied to 
military M&S (e.g., the Unified Modeling 
Language, or UML, for conceptual 
modeling). There are nearly as many 
governance mechanisms for these 
standards as there are standards, ranging 
from the highly formal (e.g., IEEE standards 
with explicitly defined community voting 
procedures for standards revisions) to the 
highly informal (e.g., some military 
standards with appointed panels of users, 
technical experts, and sponsors deciding on 
proposed revisions). 

Beyond governance formality, M&S 
standards also vary by degree of technical 
specificity, defined as the extent to which a 
particular technical utilization or solution is 
mandated by the standard. In this attribute 
the standards range from very low 
specificity, such as broad guidelines to 
users (e.g., diagram formats in UML), to 
very high technical specificity, such as 
common software components required for 
all users (e.g., interoperability middleware in 
TENA). It has been conjectured that these 
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attributes of standards, governance 
formality and technical specificity, perhaps 
combined with other attributes can be 
shown to be correlated with or even 
predictive of the expected utility and ubiquity 
of standards for military M&S. 

The DoD’s Modeling and Simulation 
Coordination Office (MSCO) is currently 
engaged in a comprehensive survey of 
military M&S standards. An outcome of the 
survey will be the cataloging and 
characterization of military M&S standards 
on many attributes of interest. This effort 
will enable standards developers and users 
to understand what standards are available 
to support ongoing and planned military 
M&S development projects and to identify 
gaps where new standards might be 
beneficial. The mechanisms by which 
standards in general, and military M&S 
standards in particular are created and 
maintained vary widely. An understanding 
of the processes available can guide the 
selection of the appropriate venue and 
process for introducing a new proposed 
standard. 

One of the most important and successful 
M&S standards of recent years is the High 
Level Architecture (HLA), which was derived 
from the Distributed Interactive Simulation 
(DIS) standard during the early nineties 
(Hollenbach, 2009). 

4.0 DISCUSSION 

During the workshop, several topic areas 
relating to standards were discussed. In 
this section, a review of some of the key 
topics is given. 

4.1 Measuring M&S standards 

The idea of considering standards success 
as a research question, and associating that 
success with specific standards attributes, 
was well received. Return on Investment 
(ROI) has been the type of measure of a 
standard’s worth due to its implications on 
cost, as opposed to quality and reliability 
improvements. However, it is hard to define 
ROI for an M&S standard as it is difficult to 


know the impact if the standard was not 
implemented, therefore ROI should not be 
the only focus of a standards worth. 

Other possible measures include the value 
that a standard brings to a simulation, the 
technology advancement it enables, the 
project risk it mitigates and the impact on 
the M&S community. 

4.2 Confusion 

Within the M&S world, there is a lot of 
confusion relating the terminology used. 

For instance, the terms ‘modeling’ and 
‘simulation’ are often used interchangeably 
even though they are distinct concepts. 
Some definitions for these terms is given 
below: 

* Model - A physical, mathematical, or 
otherwise logical representation of a 
system, entity, phenomenon, or process. 
[DOD, 1998] 

* Simulation - A method for implementing a 
model over time. Also, a technique for 
testing, analysis, or training in which real 
world systems are used, or where a model 
reproduces real world and conceptual 
systems. [DOD, 1998] 

Other terms that are confused include 
Fidelity (the accuracy of model’s 
representation or simulation’s results) and 
resolution (the degree of detail with which 
the real-world is simulated). An example of 
a high resolution but low fidelity simulation 
would be Microsoft flight simulator. 

Another example is composability and 
interoperability. Composability is different 
from interoperability as it implies models 
working together to produce a valid whole 
meta-model. Interoperability simply implies 
that two simulations are able to 
communicate with each other. 

To understand this difference, consider a 
‘fish tank’ simulation and a combat 
simulation. Both simulations might have an 
object within them called ‘tank’, and they 
might be able to interoperate by passing the 
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‘tank’ object to each other. However, the 
concept of the ‘tank’ object is completely 
different in each model and thus the models 
are not composable. 

The challenge of composability brings to 
M&S are some of the most difficult ones that 
M&S research have faced in recent years. 

4.3 Education 

The complexity of M&S is likely to increase 
in the coming years and this will increase 
the need for more education of M&S and 
standards as well. Given the confusion 
associated with M&S terminology, this 
education should apply to customers of 
M&S as well as the next generation of 
professionals. 

It is, however, important to remember that 
users are not interested in M&S standards; 
they are interested in the functionality that 
happens because of standards. 

Standards themselves are not a new 
concept and they already impact on every 
aspect of our lives. There was no 
consensus within the workshop whether it 
was appropriate to compare M&S standards 
to other existing standards though several 
useful standards analogies were given. It 
was clear that standards do have a lifecycle 
and it was suggested that we should focus 
our efforts on the standards that are likely to 
have the longest lifecycles. This focus 
might help mitigate some of the ‘lag effect’ 
that standards tend to have compared to 
cutting edge technology. 

5.0 CONCLUSION 

The future of M&S will see a changing shift 
from construction simulation to live and 
virtual simulation. With this shift come lots 
of standards requirements from new 
problems and areas that are appearing i.e. 
VV&A standards and data standards. The 
future also holds many challenges for M&S 
including composability and determining the 
ROI from M&S standards. 


occurred. Therefore, it is difficult to say 
what M&S will be like in 10 years, it might 
be similar to the situation of today or a 
completely unimaginable area. 

The workshop did not cover all aspects of 
standards and there are many more 
discussion areas that need to be addressed. 
The next workshop in the series was held 
on, which will be on M&S standards 
governance, was held on August 4 th to 6 th 
2010 . 

Return your imagination back to the 
caveman’s children and their throwing of 
stones at the boulder, which was discussed 
at the start of this paper. After a while of the 
playing this game, the children would, no 
doubt, impose a limit on how close you were 
allowed to get to the boulder before 
throwing stone because otherwise the game 
would become too easy. This would have 
been the first M&S standard to be 
developed as it is part of human nature to 
organize things. Thus the question of future 
M&S standards should not be about ‘if’ they 
are going to occur but instead it should be 
about 'when they will occur’, ‘by whom’ and 
‘how’, 
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